Papers by Mitesh M. Khapra
IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languages (2022.emnlp-main)
Copied to clipboard
Aman Kumar, Himani Shrotriya, Prachi Sahu, Amogh Mishra, Raj Dabre, Ratish Puduppully, Anoop Kunchukuttan, Mitesh M. Khapra, Pratyush Kumar
| Challenge: | IndicNLG is a non-English language that is hampered by the scarcity of datasets. |
| Approach: | They propose to create a dataset for natural language generation for 11 Indic languages . they use a set of pre-trained models to train multilingual models . |
| Outcome: | The proposed datasets show that pre-trained models perform well in multilingual and monolingual tasks. |
A Mixed Hierarchical Attention Based Encoder-Decoder Approach for Standard Table Summarization (N18-2)
Copied to clipboard
| Challenge: | Structured data summarization involves generation of summaries from structured input data. |
| Approach: | They propose a hierarchical attention-based encoder-decoder model which leverages the structure in addition to the content of the tables. |
| Outcome: | The proposed model improves on the weathergov dataset by 30% over the current state-of-the-art. |
Towards a Better Metric for Evaluating Question Generation Systems (D18-1)
Copied to clipboard
| Challenge: | Existing evaluation metrics based on n-gram similarity do not correlate well with human judgments . large datasets for document Question Answering (QA) have enabled the development of end-to-end supervised models . |
| Approach: | They propose a scoring function to capture answerability of questions . they also integrate existing similarity metrics into the scoring function . |
| Outcome: | The proposed scoring function improves human judgments on question answerability . the proposed scoring functions are made publicly available . |
Joint Transformer/RNN Architecture for Gesture Typing in Indic Languages (2020.coling-main)
Copied to clipboard
| Challenge: | Gesture typing is a method of typing words on a touch-based keyboard by drawing a continuous trace passing through the relevant keys. |
| Approach: | They propose a keyboard that supports gesture typing in Indic languages by drawing a continuous trace over the keyboard and the finger needs to be lifted only once a word is completed. |
| Outcome: | The proposed model performs path decoding, transliteration and transliterations correction. |
Naamapadam: A Large-Scale Named Entity Annotated Data for Indic Languages (2023.acl-long)
Copied to clipboard
Arnav Mhaske, Harshit Kedia, Sumanth Doddapaneni, Mitesh M. Khapra, Pratyush Kumar, Rudra Murthy, Anoop Kunchukuttan
| Challenge: | Named Entity Recognition (NER) is a fundamental task in natural language processing (NLP). |
| Approach: | They present the largest publicly available Named Entity Recognition dataset for the 11 major Indian languages from two language families. |
| Outcome: | The proposed dataset is the largest publicly available Named Entity Recognition (NER) dataset for the 11 major Indian languages from two language families. |
IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages (2020.findings-emnlp)
Copied to clipboard
Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, Pratyush Kumar
| Challenge: | In this paper, we present NLP resources for 11 major Indian languages . distributional representations are the cornerstone of modern NLP, authors say . |
| Approach: | They introduce NLP resources for 11 major Indian languages from two major language families . monolingual corpora contains 8.8 billion tokens across all 11 languages and Indian English . they also compile a benchmark for Indian language NLU to evaluate their results . |
| Outcome: | The monolingual corpora contains 8.8 billion tokens across all 11 languages and Indian English . the pre-trained language models are based on the compact ALBERT model . |
A Tutorial on Evaluation Metrics used in Natural Language Generation (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
| Approach: | This tutorial presents the evolution of automatic evaluation metrics to their current state . it aims to assess the extent of scientific progress made and identify areas/components that need improvement . |
| Outcome: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
A Dataset for Building Code-Mixed Goal Oriented Conversation Systems (C18-1)
Copied to clipboard
| Challenge: | Existing data on goal-oriented conversation systems focus on monolingual conversations and there is hardly any work on multilingual and/or code-mixed conversations. |
| Approach: | They build a goal-oriented dialog dataset containing code-mixed conversations using monolingual text from a restaurant reservation dataset. |
| Outcome: | The proposed model is based on a restaurant reservation dataset and will be made publicly available for research purposes. |
Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages (2023.acl-long)
Copied to clipboard
Sumanth Doddapaneni, Rahul Aralikatte, Gowtham Ramesh, Shreya Goyal, Mitesh M. Khapra, Anoop Kunchukuttan, Pratyush Kumar
| Challenge: | Recent advances in Natural Language Understanding are driven by pretrained multilingual models, which can potentially reduce the performance gap between high-resource languages through zero-shot knowledge transfer. |
| Approach: | They propose to create a human-supervised benchmark for Indic languages, IndicXTREME, with nine diverse NLU tasks covering 20 languages. |
| Outcome: | The proposed model improves on the monolingual corpora, IndicCorp, and IndicBERT in Indic languages with 105 evaluation sets across languages and tasks. |
Towards Exploiting Background Knowledge for Building Conversation Systems (D18-1)
Copied to clipboard
| Challenge: | Existing dialog datasets contain a sequence of utterances without any explicit background knowledge associated with them. |
| Approach: | They propose to use movie chats to generate responses by copying unstructured background knowledge . they use a dataset of 9K conversations to test whether responses are generated by copy-and-modify models . |
| Outcome: | The proposed model mimics human process of conversing by copying and/or modifying sentences from unstructured background knowledge. |
Generating Descriptions from Structured Data Using a Bifocal Attention Mechanism and Gated Orthogonalization (N18-1)
Copied to clipboard
| Challenge: | a proposed model for generating natural language descriptions is too generic and does not exploit task specific characteristics. |
| Approach: | They propose a model which uses a fused bifocal attention mechanism to exploit micro and macro level information and a gated orthogonalization mechanism to ensure that a field is remembered for a few time steps and then forgotten. |
| Outcome: | The proposed model improves on a recently released dataset with two similar datasets for French and German. |
Towards Transparent and Explainable Attention Models (2020.acl-main)
Copied to clipboard
Akash Kumar Mohankumar, Preksha Nema, Sharan Narasimhan, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran
| Challenge: | Recent studies on interpretability of attention distributions have led to notions of faithful and plausible explanations for a model’s predictions. |
| Approach: | They propose to modify LSTM cells to ensure that the hidden representations learned at different time steps are diverse. |
| Outcome: | The proposed model can provide a faithful explanation if a higher attention weight implies a greater impact on the model’s prediction. |
Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models for dialog evaluation are trained using a single relevant response and multiple random negatives. |
| Approach: | They propose a dataset to test whether model-based dialog evaluation metrics can be used to train models . they propose n-gram based metrics and embedding based ones to be used for model-driven evaluation . |
| Outcome: | The proposed model outperforms existing models on a reddit dataset on relevant responses and adversarial responses. |
Perturbation CheckLists for Evaluating NLG Evaluation Metrics (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for natural language generation are inadequate . existing metrics are not robust against simple perturbations and disagree with scores assigned by humans to perturbed output. |
| Approach: | They propose to propose checks which perturb the output and target a specific criteria and then use them to refine their evaluation. |
| Outcome: | The proposed templates show that existing evaluation metrics are not robust against simple perturbations and disagree with human scores on the perturbed output. |
On Knowledge distillation from complex networks for response prediction (N19-1)
Copied to clipboard
| Challenge: | Recent advances in Question Answering have led to the development of very complex models . however, these models are expensive in space and time and require limited resources . |
| Approach: | They propose to use simple models which learn to emulate characteristics of a teacher network . they use a 12GB Tesla K80 GPU to restrict the maximum length of the input document . |
| Outcome: | The proposed model can perform better on a Holl-E dialog dataset. |
IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation Metrics for Indian Languages (2023.acl-long)
Copied to clipboard
Ananya Sai B, Tanay Dixit, Vignesh Nagarajan, Anoop Kunchukuttan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre
| Challenge: | Recent studies on machine translation systems focus on high-resource languages, but focus has shifted to low-resourced languages. |
| Approach: | They evaluate 16 metrics from a multidimensional quality metric dataset . they show pre-trained metrics have higher correlations with annotator scores . |
| Outcome: | The proposed evaluations show that pre-trained metrics outperform COMET on Indian languages. |
Towards Interpreting BERT for Reading Comprehension Based QA (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models such as ELMO and XLNet have achieved state-of-the-art performance on various NLP tasks. |
| Approach: | They propose to define a layer’s role or functionality using Integrated Gradients and perform preliminary analysis across all layers. |
| Outcome: | The proposed model performs better than existing models on RCQA and ELMO, but it lacks the human-level performance needed to perform the task. |
Let’s Ask Again: Refine Network for Automatic Question Generation (D19-1)
Copied to clipboard
Preksha Nema, Akash Kumar Mohankumar, Mitesh M. Khapra, Balaji Vasan Srinivasan, Balaraman Ravindran
| Challenge: | Existing AQG models produce incomplete questions which look like incomplete drafts with scope for refinement. |
| Approach: | They propose a method which mimics the human process of generating questions by first creating an initial draft and then refining it. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three datasets and improves on fluency and answerability metrics. |
DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension (P18-1)
Copied to clipboard
| Challenge: | DuoRC contains 186,089 unique question-answer pairs created from 7680 movie plots . |
| Approach: | They propose a novel dataset for Reading Comprehension that motivates new challenges for neural approaches in language understanding beyond those offered by existing RC datasets. |
| Outcome: | The proposed dataset motivates several new challenges for neural approaches in language understanding beyond those offered by existing RC datasets. |
Bhasa-Abhijnaanam: Native-script and romanized Language Identification for 22 Indic languages (2023.acl-short)
Copied to clipboard
| Challenge: | Existing tools for language identification are noisy, small and similar to high-resource languages. |
| Approach: | They create a language identification test set for native-script and romanized text which spans all 22 Indic languages and train a model for romanized script. |
| Outcome: | The proposed model improves on native-script and romanized script, and is competitive or better than existing LIDs. |
On the weak link between importance and prunability of attention heads (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large fraction of attention heads can be randomly pruned with limited effect on accuracy, a new study finds . a second study finds no advantage in pruning attention heads identified to be important based on the location of a head . |
| Approach: | They examine the importance of pruning attention heads on a Transformer-based model . they find no advantage in pruning attention head positions on the BERT model based on location . |
| Outcome: | The results show that pruning strategies on Transformer and BERT models are not important based on location . the results suggest that interpretation of attention heads does not strongly inform pruning strategies. |